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UK politicians puzzle over peer review in an 
open access environment 


The UK House of Commons Science & Technology Committee is currently 
conducting an inquiry into peer review. The third public event of the inquiry was held 
on Monday 23"4 May, when the Committee heard evidence from experts on open 
access publishing and post-publication review, and from representatives of the 
research community. 


SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN 
THE EDITED QUESTIONS AND ANSWERS LISTED BELOW. THE ASTERISKED 
HEADINGS ARE MINE. 


| COMMENT ON THE SESSION AT THE BOTTOM OF THIS POST. 


UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE 
DETAILS ARE AVAILABLE HERE. 


The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for 
Ellesmere Port and Neston. Other politicians to pose the questions below were 
Graham Stringer, Labour MP for Blackley and Broughton, Roger Williams, Liberal 
Democrat MP for Brecon and Radnorshire, and Stephen Metcalfe, Conservative MP 
for South Basildon and East Thurrock. (A full list of Committee members is available 
here). 


The hearing was split into two sessions. 
== FIRST SESSION == 


Those giving evidence in the first session were Dr Rebecca Lawrence, Director, New 
Product Development at Faculty of 1000 Ltd., Mark Patterson, Director of Publishing 
at the Public Library of Science, Dr Michaela Torkar, Editorial Director at Biomed 
Central, and Dr Malcolm Read OBE, Executive Secretary of JISC. 


* On splitting traditional peer review into two separate processes: a) assessing 
a paper’s technical soundness and b) assessing its significance — a model 
pioneered by open-access publisher PLoS ONE, and now increasingly being 
adopted by traditional publishers ... 


Q162 Chair: We have heard that pre-publication peer review in most journals can be 
split, broadly, into a technical assessment and an impact assessment. Is it important 
to have both? 


Dr Torkar: ... It is fairly straightforward to think about scientific soundness because it 
should be the fundamental goal of the peer review process that we ensure all the 
publications are well controlled, that the conclusions are supported and that the study 
design is appropriate. That is fairly straightforward as a very important aspect which 
should be addressed as part of the peer review process. 


The question of the importance of impact is more difficult. When we think about high 
impact papers we think about those studies which describe findings that are far 
reaching and could influence a wide range of scientific communities and inform their 
next-stage experiments. Therefore, it is quite important to have journals that are 
selective and reach out to a broad readership, but the assessment of what is 
important can be quite subjective. That is why it is important, also, to give space to 
smaller studies that present incremental advances. Collectively, they can actually 
move fields forward in the long term. 
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Dr Patterson: ... [B]oth these tasks add something to the research communication 
process. Traditionally, technical assessment and impact assessment are wrapped up 
in a single process that happens before publication. We think there is an opportunity 
and, potentially, a lot to be gained from decoupling these two processes into 
processes best carried out before publication and those better left until after 
publication. 


One way to look at this is as follows. About 1.5 million articles are published every 
year. Before any of them are published, they are sorted into 25,000 different journals. 
So the journals are like a massive filtering and sorting process that goes on before 
publication. The question we have been thinking about is whether that is the right 
way to organise research. There are benefits to focusing on just the technical 
assessment before publication and the impact assessment after publication ... 
Online we have the opportunity to rethink, completely, how that works. Both are 
important, but we think that, potentially, they can be decoupled ... 


Dr Lawrence: ... [I]t is not known immediately how important something is. In fact, it 
takes quite a while to understand its impact. Also, what is important to some people 
may not be to others. A small piece of research may be very important if you are 
working in that key area. Therefore, the impact side of it is very subjective. 


Dr Read: ... Separating the two is important because of the time scale over which 
you get your answer. The impact is much longer. | guess the technical peer review is 
a shorter-term issue. 


* On whether in order to deliver faster publishing times it is necessary to cut 
corners by, for instance, editing papers more lightly, and whether this 
approach leads to more submissions ... 


Q167 Roger Williams: Is light copy editing a feature of how you can deliver faster 
times? 


Dr Patterson: [W]e are balancing these two competing interests of speed and 
quality. In our production process we focus on delivering really well structured files 
that will be computable, for example. We don’t expend effort in changing the 
narrative. Scientific articles aren’t works of literature. That is not to say it wouldn’t be 
nice if, sometimes, a bit more attention was paid to that. It is also true that one of the 
criteria for PLoS ONE is that the work is in intelligible English. If an editorial reviewer 
thinks that something is just not good enough and they can’t really see what is 
happening, it will be returned to the author. 


Q169 Roger Williams: Are there any other corners that your journal “cuts” in order 
to deliver faster times? 


Dr Patterson: | wouldn’t frame it that way. What we are doing is trying to identify and 
take away any unnecessary barrier to publication ... 


Q170 Roger Williams: Has your approach and the reputation you have built up 
resulted in a lot more submissions? 


Dr Patterson: PLoS ONE was launched in December 2006 and is still quite a new 
journal. It is only four and a half years’ old. We published about 4,000 articles in 2009 
and 6,700 last year, so it became the biggest peer review journal in existence in four 
years. It has grown steadily over that time ... 


* On the propagation of the PLoS ONE model and whether the journal might 
become a victim of its own success ... 


Q171 Roger Williams: Has your approach and the reputation and impact of the 
journal itself increased the number of submissions? 


Dr Patterson: It has. We see a lot of positive feedback ... the message that if | have 
a solid piece of work I’m not going to have to grapple with a journal that is basically 
biased against publication — the goal of PLoS ONE is to publish all rigorous science 
— is avery positive one which authors like. Coupled with ideas about how, then, you 
might assess the impact after publication, it is definitely gaining ground. 


The other very significant thing that has happened in the last nine to 12 months is 
that eight or more big publishers have announced PLoS ONE lookalikes, essentially. 
That is very striking. The American Institute of Physics and the American Physical 
Society have both launched physical science versions; Sage has launched a social 
science version; the BMJ group, who were actually the first, last year launched a 
clinical research version of PLoS ONE; Nature has launched a natural science 
version of PLoS ONE, and on it goes. The model is getting that level of endorsement 
from major publishers and | think, again, that is probably helping to make 
researchers very comfortable with the way in which PLoS ONE works. 
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Q172 Roger Williams: But will you be a victim of your own success? Will you be 
overwhelmed by the volume of submissions and then your time to publication suffers 
as a result? 


Dr Patterson: | certainly hope not. The growth has been pretty spectacular and has 
definitely surpassed our expectations ... 


* On whether PLoS ONE’s approach and popularity is impacting on its peer 
review process, and whether the success of its model might trigger a wider 
change in peer review ... 


Q173 Roger Williams: Do you believe that this approach has had an effect on the 
peer review process perhaps in terms of timing, quality and ease of recruiting or 
having access to reviewers? 


Dr Patterson: It is beginning to. PLoS ONE has grown very rapidly in the space of 
four years to become a very big journal. There are now another eight to 10 on the 
scene that are being launched, or are about to be launched. If another 10, 20 or 30 of 
these are launched over the next one to two years, which | think is quite likely — 
because a lot of publishers will be looking very hard and thinking that if they don’t get 
involved they will potentially lose out — that could make some fairly substantial 
changes in the way the prepublication peer review process works. There is a lot to 
say about post-publication but not yet. So | think the model could change ... 


* On whether PLoS ONE’s peer review process could be described as “light 
touch” ... 


Q176 Stephen Metcalfe: You wouldn't describe your approach as "light touch"? 


Dr Patterson: No, not at all. It is important to consider not just the peer review 
process but everything that goes on before an article is accepted for publication as 
being critical steps in quality control. There are several components to that, of which 
peer review is one. At PLoS ONE staff are involved in the first step. It goes through a 
series of quality control steps which are focused. Basically, we want to take stuff 
away from the academics so that they can focus on the science and we can sort out 
everything else. We focus on things like whether the competing interest statements 
are properly indicated; financial disclosures; if the work concerns human participants 
whether there is an ethics statement and appropriate ethical approval — a whole 
series of things like that. Hardly any manuscripts get through that without some kind 
of query going back to the author. 


Then there is a step where we involve PhD scientists who scan the work. These are 
people who have some level of subject expertise. Some — not many — of the 
submissions are rejected at that point because they are completely out of scope or 
something. They are also looking for any articles on controversial topics or anything 
that might require special treatment. They flag work like that. The work then goes to 
the editors whose responsibility it is to take on the peer review process. It is a pretty 
involved process. 


The peer review part then focuses on seven criteria to do with whether the 
methodology and analysis are appropriate; whether the conclusions are justified; 
whether the work is ethically sound and properly reported; and whether data is 
available as appropriate. There is a set of seven criteria ... 


* On cascading peer review, the difficulties in getting publishers to share 
reviews, and whether sharing is more likely in an OA environment ... 


Q181 Stephen Metcalfe: How widely used is the system of cascading submissions 
and reviews from one journal to another? 


Dr Torkar: ... We use this quite extensively at BioMed Central and, in particular, with 
the BMC series which is more or less our equivalent of PLoS ONE and was launched 
in 2001. It is a group of more than 60 community journals which are subject specific: 
BMC Immunology, BMC Genetics, etc. As they also have the premise of publishing 
all scientifically sound studies without putting too much emphasis on the impact and 
extent of the advance, they will consider manuscripts that were previously peer 
reviewed or submitted to some of our flagship journals. Sometimes the transfers will 
happen before the peer review and sometimes with the reviewers’ reports. That does 
save time for authors and reduces the burden on the peer reviewers who don’t have 
to re-review manuscripts for multiple journals. 


Dr Patterson: Cascading peer review is a phenomenon that exists at PLoS in its two 
flagship journals PLoS Biology and PLoS Medicine. Articles can be transferred from 
there to other journals. To give you a sense of the size of that, about 10% to 15% of 
submissions to PLoS ONE come from other PLoS journals. It is pretty clear that, 
internally, that works quite well. A lot of publishers think so and quite a lot of the 
evidence has shown that. 
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The much more problematic issue is the sharing of reviews from one publisher to 
another. | know you heard some talk about the Neuroscience Peer Review 
Consortium experiment which, interestingly, was not terribly popular with authors, but 
| am not sure how much publishers were really behind it. For example, it was said 
that some publishers might feel reluctant to share reviews with another journal or 
publisher because they have built up relationships with these people and there is 
some commercial value associated with that. When you hear that you have to ask 
whether that sense of ownership is in the best interests of science. | am not 
convinced ... 


... itis quite natural that journals would feel that way in a world of subscriptions 
because it is about selling a package of content to a group of readers. That is how 
the model works. Therefore, anything which allows you to improve that package of 
content is of value to you commercially. In a way, it is completely understandable that 
journals in that subscription business model would be reluctant to share their 
reviews. 


When you switch round the model, as BMC, PLoS and many others do now, in terms 
of supporting and publishing through a publication fee, considering yourselves, as 
publishers, much more as service providers — you are selling a publishing service to 
a researcher — your attitude towards sharing peer reviews might be changed. | am 
not sure. 


* On the ethics of publishing papers sponsored by pharmaceutical companies 
aiming to bring new products to the attention of doctors, whether 
pharmaceutical companies might find PLoS ONE’s peer review process more 
attractive for these purposes, and whether PLoS ONE has a financial interest in 
publishing pharma-sponsored papers ... 


Q198 Graham Stringer: ... How much commercial pressure is there from 
pharmaceutical companies to publish ... and how does that commercial pressure 
interfere with the publication? A journal that publishes a paper which means doctors 
can prescribe a particular drug stands to make a lot of money, doesn’t it? How is that 
pressure dealt with ethically? 


Dr Patterson: This is an issue which has certainly been highlighted in the evidence 
you have already heard. This is something on which, in particular, the editors of our 
journal PLoS Medicine have taken a very strong position, to reduce what they call the 
cycle of dependency in some way between the pharmaceutical industry and medical 
publishing. One of the ways in which that is manifest is with very substantial reprint 
revenues associated with high profile, hard-hitting clinical trials; for example, 
sponsored by the pharmaceutical industry. 


What PLoS Medicine and PLoS as a whole have done, in order to keep the two 
things apart and separate any commercial interest from the editorial integrity of the 
content to be published, is refuse to accept any form of drug or device advertising, 
even though it could be a significant revenue stream for us. We feel that is a very 
strong leadership position to take in that area. The business of open access is also 
very important to this. The articles we publish are open in the sense that there are no 
barriers to reusing that content. A lot of publishers retain rights to contents so that 
they can reprint the article. They are the only people who can reprint that article at 
the levels of thousands and thousands of copies for redistribution, which then earns 
them an awful lot of money. We can’t do that. 


Q199 Graham Stringer: Are you saying that reproducing your articles is free? 
Dr Patterson: Yes. 
Q200 Graham Stringer: You are very different from The Lancet or other journals? 


Dr Patterson: Totally different. We feel that is a very important principle. We have no 
unique right to take those articles and make that kind of money from them. These are 
some steps that have been taken. They are not the solution to everything, but | think 

they are important ... 


Q201 Graham Stringer: It struck me, when you spoke earlier, that if a 
pharmaceutical company wanted to get a drug to market very quickly and within the 
mindset of GPs and other doctors, your route to publication would be quicker. It 
might be an incentive, then, for them to go via a route which you said yourself — | 
can’t remember your exact words — was of a different standard; it wouldn't be sent 
back. That worried me slightly, that, commercially, it might be easier for drug 
companies to make more money by going via your route. But you don’t have a 
financial interest in that? 


Dr Patterson: There is no financial interest, in that sense. To be clear, we consider 
work that has been sponsored by the pharmaceutical industry but, obviously, it has to 
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conform to the same criteria as everything else. What might make the 
pharmaceutical industry reluctant, in terms of thinking about the value of that 
publication commercially, is that to publish in a very high prestige journal would 
probably be of great value. That is what might put them off coming to, say, PLoS 
ONE which does not, in and of itself equal high prestige ... 


* On post-publication peer review, whether publishers should incentivise 
researchers to contribute to such reviewing, what form post-publication review 
should take, and how research assessment metrics can be improved ... 


Q209 Chair: ... | want to go finally to post-publication commenting. Should 
publishers introduce some system of prestige or credit for post-publication 
commentary? Dr Patterson, why is article-based methodology a good one? | don’t 
regard it necessarily as a healthy comment if | make a speech and there is an 
endless number of blogs. No doubt | will disagree with half of them anyway. Is the 
F1000 model which uses faculty members to carry out that process a better one, or 
does it become a biased process? To finish off, let me put to all of you this question: 
what is a good system of post-publication commenting? Should there be some 
recognition of the people who participate? 


Dr Patterson: Maybe the starting point is to say that at the moment we have a very 
blunt instrument for research assessment which is basically a number — an impact 
factor — associated with a journal. We can do much better than that now. The way 
we are looking at this is to consider all the things you can potentially measure post- 
publication. It is not just about a blog comment or something like that. There is a 
whole range of metrics and indicators, including resources like Faculty of 1000, 
which can be brought to bear on the question of research assessment. Normally, 
people are looking at the research literature as a whole, they are identifying the 
papers that are important to them and they are coming to those papers. We want to 
provide an indication when they come to that paper of how important this is and what 
impact it has had through usage data, citation information, blogosphere coverage 
and social bookmarking. There are so many possibilities. 


We have moved in that direction by providing those kinds of metrics and indicators 
on every article that we publish — we are not the only people doing this but we have 
probably taken it further than most — to try to move people away from thinking about 
the merits of an article on the basis of the journal it was published in to thinking about 
the merits of the work in and of itself. Indicators and metrics can help with that. They 
aren’t the answer to the question but they will help ... 


Dr Lawrence: We would agree. Faculty of 1000 is a way of using a panel of experts. 
We have heads of faculty who then suggest the section heads who then suggest the 
faculty members. It is all very open. All their comments are against their name. On 
the question of bias, they also have to sign something to say they haven’t been 
unduly influenced and, obviously, there are issues of conflicts of interest. 


Q210 Chair: Isn’t that a more structured approach to Dr Patterson’s X Factor 
version? 


Dr Lawrence: | don’t think that any of these different metrics, on their own, are that 
strong. The point is about bringing together all the various metrics. They all have 
their own problems. To measure the impact of research you need to use different 
ones in a sensible way. In a way, the more metrics you have the better your chance 
of really understanding the impact. 


Q211 Chair: | am getting from this that your methodology is making sure that the 
judges aren't tone deaf, if | may continue to use my rotten analogy, which is a cruel 
one to you, Dr Patterson. In Dr Patterson’s case, you don’t care. 


Dr Patterson: No. To be clear, | think both approaches will be required. They are 
complementary. | would like to see — we probably will shortly — F1000 as one of the 
indicators on a PLOS article. You go to the article and say, "Ooh! It’s been highlighted 
in F1000 and this is what the person has said", or something like that. There will be a 
place for expert assessment, evaluation and organisational content post-publication, 
as well as grabbing as many metrics and indicators as you can from the world at 
large. 


== SECOND SESSION == 


Those giving evidence in the second session were Dr Janet Metcalfe, Chair of Vitae, 
Professor lan Walmsley, Pro Vice Chancellor of the University of Oxford, and 
Professor Teresa Rees CBE, Professor of Social Science and former Pro Vice 
Chancellor (Research) at Cardiff University. 


* On why researchers are under pressure to publish in high impact journals ... 


Q216 Chair: ... [Why is it that researchers are put under so much pressure to get 


work published in the high impact journals? 


Professor Walmsley: Perhaps a simple answer to that from a parochial view of a 
university person is that that is the way one’s career advances. As you heard from 
the previous panel, a lot of very good work gets published in journals that do not 
have such high visibility, and | think that is quite crucial. None the less, having a 
highly cited paper in a journal that people would regard as high profile is considered 
important as a way to raise your visibility and develop your career. 


Dr Metcalfe: We have drivers in the system, such as the research assessment 
exercise, that encourage that, so there is very strong emphasis in terms of the impact 
of the journal. Coming at it from my perspective as Vitae, it is: how do you support 
early career researchers to enter into that system and even make decisions about 
what journals they should be targeting? How do they get a sense of the most 
appropriate place for them to publish? 


Q217 Chair: Doesn't the tie-in between the research excellence framework and high 
impact journals potentially create a rather subjective judgment? 


Professor Walmsley: | would argue that the reason peer review works well is the 
expertise of the community on an inherently subjective set of criteria; that is, one can 
with any piece of work assess various objective elements of it. Is it right? Is it novel 
— that is, is it new and not been published before? But the subjective element, which 
| think differentiates a number of different journals — because they have different 
subjective criteria — is the piece that is very difficult to assess in an objective way. 
Knowing that a piece of work is going to be important is a very difficult thing to do. In 
many ways that is something best assessed post facto ... 


... Labsolutely take the point about RAE. Having sat on one of the RAE panels last 
time, | can say the panel was very clear that the forum in which the paper had been 
published was not determinative. It was reading the individual outputs and assessing 
the value of the work itself that ended up being more important. None the less, when 
a CV comes across the desk of a head of department for a faculty post, as a first 
pass through it makes a difference where those papers are published. 


* On whether researchers should be paid for their work as reviewers ... 


Q222 Roger Williams: Overall, would your judgment be that in the best of all 
possible worlds researchers should be paid for their work as peer reviewers? 


Professor Rees: | am not sure of the answer to that question. It is strange that if 
researchers do the research and publish and then do the peer review and the editing 
— in some cases now they are asked to pay for their articles to be published — one 
finds oneself responding to a memo saying, "Which journals do you think we should 
cut from the library because of budget cuts?" | would say there is a bit of a paradox 
there. 


Professor Walmsley: | would concur with that, having part of the library as my 
portfolio too. That is an internal and difficult question to address. As to whether 
reviewers should be paid, | think that may send incentives in the wrong direction. 
One wants as wide a fraction of the community with appropriate expertise to be 
involved as possible. The way we might see it internally in departments at Oxford 
would be that this is a contribution to the community, just as chairing or sitting on 
committees in the university is considered part of what you need to do in order to 
make the place and the business function. But the question is about keeping an 
appropriate lid on that. There are various ways in which one might do that, but in 
mentoring terms one would often say, "You want to review twice as many papers as 
you publish and you want to review three times as many grant applications as you 
submit." That tempers your workload and makes the whole system work. 


* On whether researchers should be formally trained to do peer review ... 


Q226 Stephen Metcalfe: ... Dr Metcalfe, | think that the number of people who take 
up the opportunity for training either in peer review or other publication training is 
relatively small. Why do you think that is? 


Dr Metcalfe: The tradition is very much an apprenticeship model. You learn the 
system by doing it in terms of writing papers, submitting them and maybe getting 
feedback from your principal investigator. Where that works it is absolutely fantastic 
in terms of somebody taking an early career researcher through the system and 
giving them feedback before they submit their articles, maybe having several 
researchers in their group giving feedback, and showing them how the whole 
process works. But, because we are a collective in terms of the academic 
community, there is opportunity for that process not to be as well supported 
throughout the whole of the academic community as it could be. 


The challenge is how to help a researcher maximise their opportunities of publication 


at submission so that they are reducing the amount of rejections and the amount of 
comments they have to do through that. Formal training in that process is one way in 
which you can do that. From some of the research Vitae has done, we have 
evidence of increases in the success rates of grant applications and fellowship 
applications by having formal training and development in working within the peer 
review systems for both of those. We could do more in advance of a researcher 
having to submit their first paper or grant proposal so that they are better informed 
and therefore more expert about how the whole process works. 


Q227 Stephen Metcalfe: You would be in favour of moving towards a more formal 
requirement for training. You consider that it should be provided across all higher 
education institutions. 


Dr Metcalfe: No, | wouldn’t go down that route. | think the opportunities to have 
training should be there. The process by which a researcher learns to become expert 
is very much up to their individual circumstances. If they are getting good individual 
nurturing and mentoring by their Pl, that is great. But there should also be the 
opportunity, for those researchers who respond more to formal training, to have that 
available as well. 


Q228 Stephen Metcalfe: Who do you think should pay for that training? 


Dr Metcalfe: Collectively, we all have a responsibility for it to work. | think journals 
have a responsibility to support and provide more information about what is required 
and to contribute to the training of their reviewers. | think institutions have a 
responsibility, as signatories to the concordat for the career development of 
researchers, to ensure that those opportunities are there. | think research and 
funding councils and Government have an obligation to provide enough funding 
within the entire system to make available that kind of training for our early career 
researchers. 


* On fraud, and what whether Oxford University has ever sacked anyone for 
fraud ... 


Q234 Graham Stringer: Last week the chair of COPE told us that if a university had 
not fired at least one academic for fraud there was something wrong with the 
university. Do you agree with that statement? Do you think she was right? If so, have 
your universities sacked any academics over the last five years for academic fraud? 


Professor Walmsley: The answer to the second question is no. 
Graham Stringer: So you are not firing. 


Professor Walmsley: Yes. | would say that the answer to the first question is 
probably no, too, but | want to be careful not to suggest that there are no ethical 
challenges within publication. We have a process within Oxford, which | am certain is 
the same at other places, to deal with that. Part of the question is: how does it come 
to your attention? 


Q235 Graham Stringer: Before you go on, is that process published? 


Professor Walmsley: Yes. It is available on the website through the Research 
Integrity Portal and there is an access through the SkillsPortal to that as well. 


Q236 Graham Stringer: So, that is true for all? 


Professor Walmsley: Yes. How do you identify and find that out? | think that 
internally, at the pre-publication end, there is great onus on researchers. As more 
and more papers are published with joint authors there is joint responsibility for doing 
that. That could lead in two directions: first, increased pressure to get it right because 
there are more people involved in the discussion; but, secondly, the chance that you 
will miss a trick or two because there are more people contributing. It is a difficult 
tension. Once the paper is out there, if an external party notes something that looks 
challenging | guess we will hear about that either from the external people or from 
editors themselves. If an editor writes, we will be able to investigate that internally. 


As to the sanction of firing someone, | said | have not known that to happen, but 
there are certainly lower levels of discipline that can happen. However, | don’t know 


what the statistics are at Oxford. 


Q237 Graham Stringer: If you were to have an investigation, would you publish the 
results? Would that become a public document? 


Professor Walmsley: | don’t know the answer to that question. 


Q238 Graham Stringer: Would you write and tell us? 


Professor Walmsley: Yes, | will do that. 
The (uncorrected) transcript of the full session is available here. 


The video of the event can be accessed here, if it does not open up 
automatically here. 


COMMENT 


Subsequent to giving evidence to MPs a slightly puzzled Rebecca Lawrence penned 
a blog post about her experience. She concluded: “I think we all left sti// wondering 
what the purpose of the whole enquiry is and what they are hoping to achieve.” 


She added: “This was made even more evident when at the end of the latest 
session, following 5 long rounds of oral sessions, MP Graham Stringer suggested 
that he felt that maybe they should have been looking at the commercial pressure on 
both editors of journals and researchers instead.” 


It’s true that the Committee gave no reason for launching the Inquiry. But there have 
been hints: In this session, for instance, MPs referred both to the so-called 
Climategate incident, and to Andrew Wakefield. 


It is also true that the Committee has at times seemed a little unfocused. But this was 
probably inevitable: opening up the topic of peer review is not unlike opening 
Pandora’s Box — all sorts of things fly out. For this reason perhaps the Committee 
has not always been able to explore in sufficient depth some of issues that have 
arisen. 


The key theme to emerge from this day of the hearing was whether peer review 
should/would change in an online, open-access environment; and if so, in what way. | 
list below some of the areas that | feel could have been examined more closely: 


e There appears to have been little discussion of the possible quality 
implications of authors (or their funders) being asked to pay a fee to 
publish in open-access journals. It is widely known (within the open- 
access movement at least) that a growing number of high-volume, low- 
quality journals are emerging that appear to offer little more than a vanity 
publishing service. | should stress that | am not referring to PLoS ONE or 
BioMed Central. | am referring to a number of start-up companies that 
have adopted the "author pays" model introduced by BioMed Central — 
and subsequently copied by PLoS ONE — but which often appear to 
make little or no effort to have the papers they publish reviewed 
effectively. The witnesses who appeared in the first part of this session 
will know of this development, and would surely have been able to talk 
through the issues. In the process the Committee could have explored the 
potential conflicts of interest (and how to overcome them) that must surely 
arise when the income of a publisher of peer-reviewed journals is directly 
related to the number of papers it accepts. 


Likewise, the Committee did not appear to address a topic that has 
become a serious problem in scholarly publishing, one that surely has 
implications for peer review. That is, although the stated purpose of 
publishing scholarly papers is to share new ideas and experimental data 
with other researchers, the emphasis today is as likely (often more likely) 
to be on furthering an author's career as it is on communicating research 
findings — which are not necessarily the same things. In other words, 
researchers are incentivised to publish as many papers as possible in 
order to maximise their chances of tenure and/or promotion. Amongst 
other things this leads to salami slicing, and inevitably to the publication of 
lower-quality, less-worthwhile papers. In a pay-to-publish environment 
that can easily morph into vanity publishing if there are inadequate 
guidelines and processes to ensure high quality this is worrying — not 
just in terms of quality in fact but, since it can cost 1,000s of dollars to 
publish a paper, in terms of costs too. 


e And while it did explore the costs of peer review, the Committee seemed 
more interested in the (very much smaller) costs of reviewers’ time, than 
the much larger, increasingly burdensome, costs of paying publishers to 
organise that peer review. And open access looks set to increase these 
costs (certainly in the short term). This issue was hinted at when Teresa 
Rees said: “It is strange that if researchers do the research and publish 
and then do the peer review and the editing — in some cases now they 
are asked to pay for their articles to be published — one finds oneself 
responding to a memo saying, ‘Which journals do you think we should cut 
from the library because of budget cuts?’ | would say there is a bit of a 
paradox there.” The Committee would have benefitted from trying to 
unpick that paradox. 


¢ Finally, the Committee did not follow through on Stephen Metcalfe’s 
questions about training researchers. As | understood it, the issue was 
whether and how researchers are taught to peer review papers. In their 
answers the witnesses focussed almost exclusively on how one trains 
researchers to get their papers through the review process, not how to 
review other researchers’ papers effectively. In the context of the inquiry 
the latter is surely the more important issue. Indeed, the fact that the 
witnesses focused on the other side of the process serves to underline 
the extent to which scholarly communication is now viewed essentially as 
a career-advancement mechanism, rather than a formal way of sharing 
research findings. As mentioned, this would seem to have implications for 
the quality of published research. At least it would have helped had the 
Committee persisted in looking at both sides of the process — after all, 
the current pressure to publish as many papers as possible leaves 
researchers vulnerable to the constant stream of email invitations from 
publishers of pay-to-publish journals whose review processes are 
obscure, and may be seriously lacking in rigour. 


| realise that, given the time available, it is probably asking too much to expect the 
Committee to explore all the issues that fly out of the peer review box, certainly in 
depth. But for an inquiry into peer review that has indicated it plans to examine, 
amongst others things, “the strengths and weaknesses of peer review as a quality 
control mechanism for scientists, publishers and the public”; “the impact of IT and 
greater use of online resources on the peer review process”; and “the processes by 
which reviewers with the requisite skills and knowledge are identified,” the above 


issues would seem to be highly germane. 


That said, Graham Stringer’s suggestion that they ought to be looking more at the 
commercial pressures faced by editors and researchers suggests that the Committee 
is beginning to understand some of the deeper forces currently influencing how peer 
review operates. 


It is also worth bearing in mind that when the Science & Technology Select 
Committee was conducting an inquiry into scientific publishing in 2004 some began 
to wonder if it had lost its way. However, when its report — Scientific Publications: 
Free for All? — was eventually published the research community quickly concluded 
that the politicians had actually understood the issues very clearly and, moreover, 
had made the correct recommendations. 


Meanwhile the current inquiry continues. Time will tell how successfully today’s crop 
of MPs can get to grips with the issues. 


Posted by Richard Poynder at 12:00 Dos] o 
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